
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



09/586,869 



06/05/2000 



22907 7590 06/01/2006 

BANNER & WITCOFF 
1001 G STREET NW 
SUITE 1100 

WASHINGTON, DC 20001 



Philip Victor Harman 



18193.16.0 



7648 



EXAMINER 



ART UNIT 



KIM, CHONG R 

— r 



PAPER NUMBER 



2624 



DATE MAILED: 06/01/2006 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 


Application No. 

09/586,869 


Applicant(s) 

HARMAN, PHILIP VICTOR 


Examiner 

Charles Kim 


Art Unit 
2624 





~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS. 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment See 37 CFR 1.704(b). 

Status 

1)^ Responsive to communication(s) filed on 06 March 2006 . 
2a)^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-14. 18-23,27-35 and 43-51 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) E3 Claim(s) 45-51 is/are allowed. 

6) KI Claim(s) 1-3.5-14. 18.23.27-33,43 and 44 is/are rejected. 

7) E3 Claim(s) 4. 19-22.34 and 35 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 05 June 2000 is/are: a)S accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
aM All b)D Some * c)D None of: 

1 .(3 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) CD Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 ) d Notice of Informal Patent Application (PTO-152) 

Paper No(s)/Mail Date . 6) □ Other: . 
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DETAILED ACTION 



Response to Amendment and Arguments 

1 . Applicant's amendment filed on March 6, 2006 has been entered and made of record. 

2. In view of applicant's amendment, the 1 12 second paragraph rejections are withdrawn. 

3. Applicant's arguments have been fully considered, but they are not deemed to be 
persuasive for at least the following reasons. 

Applicants argue (page 9) that their claimed invention (claims 1 and 27) differ from the 
prior art because "Kawabata lacks a teaching or suggestion of the claim 1 combination of 
features including identifying at least one object within a 2D image without using distance 
measurement data and allocating a depth tag to the at least one object." More specifically, 
applicants allege that Kawabata "requires the depth data (or distance information) to be known 
before the object is even identified." The examiner disagrees. Kawabata explains that an object 
O is identified in a 30X20 original image (see the grey portion in figure 2A), wherein the original 
image is obtained from CCD imaging devices (3, 4) [col. 4, lines 40-52]. The image is divided 
into 6X4 blocks and distance calculations are performed on the 6X4 blocks. This reduces the 
number of distance calculations from 600 (30X20) to 24 (6X4) (col. 6, lines 18-30). 
Accordingly, the distance measurement data is determined after the object is identified (col. 25- 
53 and figures 2B-2C). Hence, Kawabata teaches identifying the object within a 2D image 
without using distance measurement data, as recited in amended claims 1 and 27. 
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Claim Rejections - 35 USC § 102 
The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

4. Claims 1-3, 13-14, 27, 32, 33, 43-44 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Kawabata, U.S. Patent No. 6,370,262 ("Kawabata"). 

1 . Referring to claim 1 , Kawabata discloses a method of producing a depth map comprising 
the steps of: 

a. identifying at least one object within a 2D image (col. 6, lines 19-21 and figure 

2A); 

b. allocating an identifying tag (address) to the at least one object (col. 6, lines 48- 

53); 

c. allocating a depth tag to the at least one object (col. 6, lines 21-24 and figure 2B); 

d. determining and defining an outline (contour) of the at least one object (col. 6, 
lines 40-48); and 

e. encoding the identifying tag, the depth tag and the outline, of the at least one 
object to produce a depth map (col. 6, lines 19-53. Note that the depth map in figure 2C is 
produced based on the encoding of the identifying tag, depth tag, and outline of the object). 
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Referring to claim 2, Kawabata further discloses that the object outline is defined by a 
series of co-ordinates (col. 6, lines 44-48). 

Referring to claim 3, Kawabata further discloses that the identifying tag is a unique 
number (col. 6, lines 48-53). 

Referring to claim 13, Kawabata further discloses that the depth tag is a numerical value 
(figure 2B). 

Referring to claim 14, Kawabata further discloses that the numerical value ranges from 0 
to 255 (col. 6, lines 21-24 and figure 2B). 

Referring to claim 27, Kawabata discloses a method of encoding a depth map including: 

a. allocating an object identifier (address) to an object (col. 6, lines 48-53); 

b. allocating a depth tag to the object (col. 6, lines 21-24 and figure 2B); 

c. defining an outline (contour) of the object (col. 6, lines 40-48); and 

d. producing a depth map by encoding the depth tag and the outline of the object 
(col. 6, lines 19-53. Note that the depth map in figure 2C is produced based on the encoding of 
the depth tag, and outline of the object). 

Referring to claim 32, Kawabata further discloses that the object outline is defined by at 
least one geometric shape (figure 2C). 

Referring to claim 33, Kawabata further discloses that the geometric shape is defined by 
the form of the shape and the parameters of the shape (col. 6, lines 31-53). 

Referring to claim 43, Kawabata further discloses a method of converting 2D images into 
stereoscopic images applying a depth map generated above (claim 1) [col. 5, lines 8-15, col. 6, 
lines 18-64]. 
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Referring to claim 44, further discloses a method of converting 2D images into 
stereoscopic images applying a depth map generated above (claim 27) [col. 5, lines 8-15, col. 6, 
lines 18-64]. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over the combination of 
Kawabata, U.S. Patent No. 6,370,262 ("Kawabata") and Matsugu et al., U.S. Patent No. 
6,167,167 ("Matsugu"). 

Referring to claim 5, Kawabata does not explicitly disclose that the step of determining 
the outline further includes tracing the at least one object pixel by pixel. However, this feature 
was exceedingly well known in the art. For example, Matsugu discloses the step of determining 
an outline of an object by tracing the object pixel by pixel (col. 15, line 35-col. 16, line 61). 

Kawabata and Matsugu are combinable because they are both concerned with image 
processing methods that determine the outline of an object. At the time of the invention, it 
would have been obvious to a person of ordinary skill in the art to modify Kawabata' s outline 
determining step in view of Matsugu. The suggestion/motivation for doing so would have been 
enhance the accuracy of the outline detection process (Matsugu, col. 3, line 4-6). Therefore, it 
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would have been obvious to combine Kawabata with Matsugu to obtain the invention as 
specified in claim 5. 

6. Claims 6-10, 28-31 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Kawabata, U.S. Patent No. 6,370,262 ("Kawabata") and Meek et al. U.S. Patent 
No. 6,029,173 ("Meek"). 

Referring to claims 6-8, Kawabata does not explicitly disclose that the step of 
determining the outline further includes using straight lines, curve approximations, or Bezier 
curves to approximate the outline of the at least one object. However, this feature was 
exceedingly well known in the art. For example, Meek discloses a step of determining an outline 
further that includes using straight line, curve, and Bezier curve approximations to approximate 
the outline of the at least one object (column 6, line 20-30). It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to use straight line, curve, and 
Bezier curve approximations to approximate an outline as taught by Meek in order to minimize 
the storage requirements while providing a high level of accuracy in the representation of other- 
than-straight (curved) features (Meek, column 4, line 32-46). 

Referring to claim 9, Kawabata does not explicitly disclose that the step of determining 
the outline further includes comparing the object with a library of curves and/or generic 
geometric shapes to approximate the outline. However, this feature was exceedingly well known 
in the art. For example, Meek discloses a method and system for representation and use of shape 
information in geographic databases wherein a step of determining an outline (shape) includes 
comparing (matching) the object with a library of curves (column 8, line 50-64) and/or generic or 
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geometric shapes to approximate the outline (column 6, line 14-30; column 8, line 20-48). It 
would have been obvious to one of ordinary skill in the art at the time the invention was made to 
compare the object with a library of curves and/or generic geometric shapes as taught by Meek in 
order to approximate the outline and minimize the storage requirements while providing a high 
level of accuracy in the representation of other-than-straight (curved) features (Meek, column 4, 
line 32-46). 

Referring to claim 10, Meek further discloses that the curve and/or generic or geometric 
shape are scaled to best fit the object (column 8, line 50-64). 

Referring to claim 28, Kawabata does not explicitly disclose that the object outline is 
defined by a series of x, y coordinates, each x, y coordinate being separated by a curve. 
However, this feature was exceedingly well known in the art. For example, Meek discloses an 
object outline that is defined by a series of x, y coordinates, each x, y coordinate being separated 
by a curve (figure 5). Therefore, it would have been obvious to combine Kawabata and Meek, 
for the reasons stated above. 

Referring to claim 29, Meek discloses further that each curve (other-than-straight 
segment) is stored in a library and allocated a unique numerical number (index reference value; 
column 8, line 60-64). 

Referring to claim 30, Meek further discloses that said object outline also includes data 
on the orientation (rotation) of each curve (column 8, line 50-64). 

Referring to claim 31, Meek discloses that each said curve is a Bezier curve (column 6, 
line 14-30). 
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7. Claims 1 1-12 are rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Kawabata, U.S. Patent No. 6,370,262 ("Kawabata") and Nourbakhsh et al., U.S. 
Patent No. 5,793,900 ("Nourbakhsh."). 

Referring to claims 11-12, Kawabata does not disclose that the depth tag includes a color 
code, wherein white represents one of objects relatively close to the viewer, or objects relatively 
distant from the viewer and black represents the other. However, this feature was exceedingly 
well known in the art. For example, Nourbakhsh discloses generating categorical depth maps 
using passive defocus sensing wherein a depth map is an array of categorical depth values, each 
value indicating the depth of the scene for a given region such that depth values of 2, 1, and 0 
correspond to close, medium, and far, respectively (column 5, line 9-15). Nourbakhsh further 
discloses that close regions are lightly shaded, medium regions are medium shaded, and far 
regions are darkly shaded (Figures 2-7; column 5, line 20-25). It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to represent objects relatively 
close to the viewer as white, and objects relatively distant from the viewer with black as taught 
by Nourbakhsh in order to give the viewer an impression of depth using varying pixel intensities 
since a brighter portion logically indicates a closer portion which is easier to see, and a darker 
portion indicates a distant portion which is more difficult to see (Nourbakhsh, column 5, line 20- 
31). 



8. Claim 18 is rejected under 35 U.S.C. 103(a) as being unpatentable over the combination 
of Kawabata, U.S. Patent No. 6,370,262 ("Kawabata") and Eleftheriadis et al, U.S. Patent No. 
6,055,330 ("Eleftheriadis"). 
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Referring to claim 18, Kawabata does not explicitly disclose the step of tracking the at 
least one object on successive frames of the image, and determining and assigning depth tags for 
the at least one object in each respective frame. However, this feature was exceedingly well 
known in the art. For example, Eleftheriadis discloses the step of tracking at least one object on 
successive frames of an image, and determining and assigning depth tags for the at least one 
object in each respective frame (column 18, line 45-55. Eleftheriadis explains that the "objects 
are adequately tracked" on successive frames). 

Kawabata and Eleftheriadis are combinable because they are both concerned with image 
processing systems for producing depth maps. At the time of the invention, it would have been 
obvious to a person of ordinary skill in the art to modify the method of Kawabata in view of 
Eleftheriadis. The suggestion/motivation for doing so would have been to enhance the flexibility 
of the system. Therefore, it would have been obvious to combine Kawabata with Eleftheriadis to 
obtain the invention as specified in claim 18. 

9. Claims 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over Kawabata, U.S. 
Patent No. 6,370,262 ("Kawabata"). 

Referring to claim 23, Kawabata does not explicitly disclose the step of producing 
grayscale images that are at a lower resolution than the 2D image. However, Official notice is 
taken that producing images at a lower resolution was exceedingly well known in the art. 
Therefore, it would have been obvious to modify Kawabata' s method to include the step of 
producing grayscale images that are at a lower resolution than the 2D image in order to enhance 
the efficiency of the image storage/transmission process. 
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Allowable Subject Matter 

10. Claims 45-51 are allowed. 

1 1 . Claims 4, 19-22, 34-35 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 



Conclusion 

12. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Charles Kim whose telephone number is 571-272-7421. The 
examiner can normally be reached on Mon thru Thurs 8:30am to 6pm and alternating Fri 9:30am 
to 6pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on 571-272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-272-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

ck 

May 17, 2006 




